Selecting a Device to Display Content

ABSTRACT

A method for displaying content is described. The method includes sending multimedia content to a first device and a second device for display on the first device and the second device; selecting one of the first device and the second device to display additional content based on a type of the additional content or a source of the additional content; sending the additional content to the selected device for display on the selected device.

CROSS-REFERENCE TO RELATED APPLICATIONS; BENEFIT CLAIM

This application is a continuation of U.S. application Ser. No.12/693,410, filed Jan. 25, 2010, the entire contents of which is herebyincorporated by reference as if fully set forth herein. The applicant(s)hereby rescind any disclaimer of claim scope in the parentapplication(s) or the prosecution history thereof and advise the USPTOthat the claims in this application may be broader than any claim in theparent application(s).

FIELD OF THE INVENTION

The present invention relates to user interface devices. Specifically,the invention relates to concurrent use of multiple user interfacedevices.

BACKGROUND

The approaches described in this section are approaches that could bepursued, but not necessarily approaches that have been previouslyconceived or pursued. Therefore, unless otherwise indicated, it shouldnot be assumed that any of the approaches described in this sectionqualify as prior art merely by virtue of their inclusion in thissection.

In general user interface systems found in televisions, laptops,tablets, phones, kiosks, or most other devices include a display screenand an interactive interface. The interactive interface may includephysical control buttons (e.g., buttons found on remote controls, amouse, a joystick, a keyboard, etc.).

As shown in FIG. 1, in some systems, a touch screen remote control (10)may be used to operate a media device (e.g., video cassette recorder(VCR) (20)) which outputs media content (35) displayed on a separatedisplay screen (30). The remote control (10) executes a remote controloperating system and displays a touch screen menu (15) specificallydesigned for and displayed exclusively on the remote control (10). Thecommunicatively coupled media device (20) receives user commandssubmitted to the remote control (10) and displays media content (35)(e.g., a movie or a show) based on the user command selected on theremote control (10).

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings and in whichlike reference numerals refer to similar elements and in which:

FIG. 1 is a block diagram illustrating a prior art system;

FIG. 2A is a block diagram illustrating an example system in accordancewith one or more embodiments;

FIG. 2B is a block diagram illustrating an example user interface devicein accordance with one or more embodiments;

FIGS. 3A-3G illustrate example screen shots in accordance with one ormore embodiments;

FIG. 4 shows a block diagram that illustrates a system upon which anembodiment of the invention may be implemented.

DETAILED DESCRIPTION

In the following description, for the purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the present invention. It will be apparent, however,that the present invention may be practiced without these specificdetails. In other instances, well-known structures and devices are shownin block diagram form in order to avoid unnecessarily obscuring thepresent invention.

Several features are described hereafter that can each be usedindependently of one another or with any combination of the otherfeatures. However, any individual feature might not address any of theproblems discussed above or might only address one of the problemsdiscussed above. Some of the problems discussed above might not be fullyaddressed by any of the features described herein. Although headings areprovided, information related to a particular heading, but not found inthe section having that heading, may also be found elsewhere in thespecification.

Example features are described according to the following outline:

-   -   1.0 FUNCTIONAL OVERVIEW    -   2.0 SYSTEM ARCHITECTURE    -   3.1 CONTENT MANAGEMENT—SIMILAR AND NON-SIMILAR DISPLAYS    -   3.2 CONTENT MANAGEMENT—SPATIAL AND TEMPORAL SPANNING    -   3.3 CONTENT MANAGEMENT—ALTERNATE CONTENT    -   3.4 CONTENT MANAGEMENT—INFORMATIONAL CONTENT/SPONSTORED CONTENT    -   3.5 CONTENT MANAGEMENT—ALERTS/NOTIFICATIONS    -   3.6 CONTENT MANAGEMENT—TIME DELAY    -   3.7 CONTENT MANAGEMENT—IMAGE LAYER SELECTION    -   3.8 CONTENT MANAGEMENT—AUDIO    -   3.9 CONTENT MANAGEMENT—MULTIPLE SECONDARY USER INTERFACE DEVICES    -   4.0 USER INTERFACE DEVICE—EXAMPLE IMPLEMENTATION    -   5.0 COMMAND EXECUTION—EXAMPLE IMPLEMENATION    -   6.0 HARDWARE OVERVIEW    -   6.0 EXTENSIONS AND ALTERNATIVES

1.0 Functional Overview

In an embodiment, a method includes: transmitting multimedia content fordisplay on a first user interface device; concurrently transmittingrelated content for display on a second user interface device, at leasta portion of the multimedia content and at least a portion of therelated content being similar or identical; obtaining user inputreceived at the second user interface device; performing an operationassociated with the multimedia content displayed on the first userinterface device, based on the user input received at the second userinterface device; wherein the method is performed by at least one devicecomprising a processor.

Transmitting the multimedia content for display on the first userinterface and concurrently transmitting the related content for displayon the second user interface may be performed by the same device.

The method may include selecting a spatial portion of a frame in themultimedia content as a whole frame in the related content. The spatialportion of the frame in the multimedia content may be selected as thewhole frame in the related content in response to detecting that thespatial portion comprises a menu. The spatial portion of the frame inthe multimedia content may be selected as the whole frame in the relatedcontent based on the user input.

The related content may include the multimedia content with additionalcontent overlaid on the multimedia content. The related content mayinclude an alternate version of the multimedia content displayed on thefirst user interface device. The related content may includeadvertisements for products or services within the multimedia contentdisplayed on the first user interface device. The related content mayinclude information describing one or more image attributes within themultimedia content displayed on the first user interface device.

In an embodiment, a method includes: causing displaying of a first menuon the first user interface device; transmitting a second menu to asecond user interface device for display on the second user interfacedevice, at least a portion of the first menu and at least a portion ofthe menu being similar or identical; obtaining a menu selection receivedat the second user interface device; performing an operation associatedwith multimedia content displayed on the first user interface device,based on the menu selection received at the second user interfacedevice; wherein the method is performed by at least one devicecomprising a processor. The first user interface device may be anon-touch screen interface and the second user interface device may be atouch screen interface.

In an embodiment, a method includes: transmitting multimedia content toa first user interface device and a second user interface device forconcurrent display; selecting one of the first user interface device andthe second user interface device to display additional content;transmitting the additional content to the selected user interfacedevice for display on the selected user interface device. The additionalcontent may be overlaid on the multimedia content for display on theselected user interface device. The additional content may be displayedin response to receiving a request for the additional content on theselected user interface device.

In an embodiment, the additional content may include an alertnotification. The alert notification may be displayed on the first userinterface device and detailed information associated with the alertnotification is displayed on the second user interface device. Theselecting step may include selecting the second user interface device,and the method may include subsequent to a specified time interval ofnot receiving user input at the second user interface device in responseto displaying the alert notification on the second user interfacedevice: displaying the alert notification on the first user interfacedevice.

One of the first user interface device and the second user interfacedevice may be selected based on a type of the additional content or asource of the additional content.

In an embodiment, the method includes: displaying video contentcomprising a plurality of frames, each frame overlaid with a menu on afirst user interface device; concurrently with displaying the videocontent on the first user interface device, displaying a single frame ofthe plurality of frames overlaid with the menu, on a second userinterface device; receiving user input at the second user interfacedevice; performing an operation associated with the video contentdisplayed on the first user interface device based on the user inputreceived at the second user interface device. The video content and thesingle frame from the video content may be received by the first userinterface device and the second user interface device, respectively,from a same content source.

In an embodiment, a method includes: a media device transmittingmultimedia content to a user interface for displaying of the multimediacontent on the television; the media device transmitting the multimediacontent to a cellular telephone for displaying of the multimedia contenton the cellular telephone; the user interface and the cellular telephoneconcurrently displaying the multimedia content received from the mediadevice. At least one of the frame rate and the resolution for displayingthe multimedia content on the user interface may be different than theframe rate and the resolution for displaying the multimedia content onthe cellular telephone.

In an embodiment, a method includes: transmitting multimedia content toa first user interface device for display on the first user interfacedevice; concurrently with transmitting the multimedia content to thefirst user interface device, receiving a first message associated with auser from a web server; transmitting information associated with thefirst message to a second user interface device for display on thesecond user interface device; wherein the method is performed by atleast one device comprising a processor. The second user interfacedevice may be a cellular telephone displaying a text message based onthe information associated with the first message. The method mayfurther include receiving a second message from the second userinterface device based on user input and transmitting informationassociated with the second message to the web server.

In an embodiment, a method includes: playing multimedia content only ona first user interface device of a first user interface device and asecond user interface device; detecting that the second user interfacedevice has moved outside of the specified distance from the first userinterface device; in response to detecting that the second userinterface device has moved to outside of the specified distance, playingthe multimedia content on the second user interface device. The methodmay further include: in response to detecting that the second userinterface device has moved to outside of the specified distance,stopping playing of the multimedia content on the first user interfacedevice. The method may further include subsequent to the playing themultimedia content on the second user interface device, detecting thatthe second user interface device has moved back within the specifieddistance from the first user interface device; in response to detectingthat the second user interface device is moved back within the specifieddistance from the first user interface device, stopping playing of themultimedia content on the second user interface device.

Although specific components are recited herein as performing the methodsteps, in other embodiments agents or mechanisms acting on behalf of thespecified components may perform the method steps. Further, althoughsome aspects of the invention are discussed with respect to componentson a system, the invention may be implemented with componentsdistributed over multiple systems. Embodiments of the invention alsoinclude any system that includes the means for performing the methodsteps described herein. Embodiments of the invention also include acomputer readable medium with instructions, which when executed, causethe method steps described herein to be performed.

2.0 System Architecture

Although a specific computer architecture is described herein, otherembodiments of the invention are applicable to any architecture that canbe used to perform the functions described herein.

FIG. 1 shows an example system in accordance with one or moreembodiments. The content system (100) includes one or more media devicesthat function as content sources (110), user interface devices (115),and/or content management devices (130). Each of these components arepresented to clarify the functionalities described herein and may not benecessary to implement the invention. Furthermore, components not shownin FIG. 1 may also be used to perform the functionalities describedherein. Functionalities described as performed by one component mayinstead be performed by another component.

In an embodiment, the content system (100) may include media devicesthat function as content sources (110). Content sources (110) generallyrepresent any source of audio and/or visual content. Examples of contentsources (110) may include a Digital Video Disc player that reads datafrom a Digital Video Disc (DVD) or a video cassette recorder (VCR) thatreads data from a video cassette. Other examples include a digital videorecorder, a set-top box, a computer system, a media device, a localserver, a web server, a data repository, a kiosk, a mobile device, orany other source of content. The content system (100) may also receivecontent from other content systems. The content system (100) may includeone or more components which allow the content system to (100) receiveand/or transmit content. For example, the content system (100) mayinclude a network card, a tuner, a compressor, a de-compressor, a modem,an encryption device, a de-encryption device, a multiplexer, ademultiplexer, a receiver, or any component involved in receiving ortransmitting data. In an embodiment, the content system (100) mayreceive and/or transmit content over wired and/or wireless segments. Forexample, the content system (100) may receive content on a broadcaststream, a network stream (e.g., internet, intranet, local area network),a Bluetooth signal, an infrared signal, any suitable frequency in theelectro-magnetic spectrum and/or via any other accessible method ofreceiving content from or transmitting content to devices within thecontent system (100) or outside of the content system (100).

In an embodiment, the content system (100) may include media devicesthat function as user interface devices (115). A user interface device(115) generally represents any device with input and/or output means. Auser interface device (115) may include one or more of: a displayscreen, a touch screen interface, a keypad, a mouse, a joystick, ascanner, a speaker, an audio input, an audio output, a camera, etc.Examples of user interface devices (115) include monitors, televisionsets, projectors, mobile device interfaces, kiosks, tablets, laptops,speakers, headphones or any other devices which may be used to receiveand/or present audio content and/or visual content. In an embodiment, atleast one of the user interface devices (115) within the content system(100) may be configured to receive input from a user.

In an embodiment, a user interface device (115) may be configured toautomatically detect user interaction elements in a display. Forexample, a secondary user interface device may be configured to displaya menu concurrently displayed on a primary user interface device. Thesecondary user interface device may automatically identify “buttons”within the menu as user interaction elements (e.g., by optical characterrecognition, button shape recognition, color based identification,etc.). The secondary user interface device may obtain identify userinteraction elements displayed on screen based on data identifying theuser interaction elements. For example, the x and y coordinates on adisplay identifying the corners of a button may be received. Anotherexample, may involve receiving the images or features of the buttonsseparately that can be compared to a menu screen to identify matchingelements. The matching elements may then be identified as userinteraction elements.

The user interaction elements may be visually indicated to a user. Forexample, the user interaction elements may be overlaid with specialshading, colors, lines, or other suitable visual clues which canidentify the user interaction elements. The secondary user interface maythen allow a user to select the automatically identified buttons. Forexample, on a touch screen secondary user interface, the automaticallyidentified buttons may be touched by a user to select the buttons.Alternatively, a keypad (or other input means) on the secondary userinterface may be used to select the automatically identified buttonsdisplayed on the secondary user interface. In an embodiment, the userinterface device may detect text input fields (e.g., by detecting whiteboxes or a cursor in displayed content), a scroll bar (e.g., bydetecting opposing arrows along a vertical column in displayed content),a radio button selection (e.g., by detecting a set of verticallyoriented circles or squares in displayed content), or any otherinterface components that may be used to obtain user input. Thesecondary user interface may be configured to transmit informationassociated with user input to another component (e.g., the contentmanagement device (130)) of the content system (100).

In an embodiment, different types of user interface devices (115) may beused concurrently in the content system (100). Different resolutionsand/or frame rates may be used for displaying content on different userinterface devices. In an embodiment, one user interface device may bereferred to herein as a primary user interface device and another userinterface device may be a secondary or ancillary user interface devicefor clarity. However, the functionality described herein pertaining to aparticular user interface device may be applicable to another userinterface device. Accordingly, reference to a primary user interfacedevice or a secondary user interface device should not be construed aslimiting in scope.

3.1 Content Management—Related Content

In an embodiment, the content system (100) may include media devicesthat function as content management devices (130). Content managementdevices (130) determine what content is to be played on each userinterface device of a set of user interface devices. For example, acontent management device (130) may be configured to display mediacontent concurrently on a primary user interface device and a secondaryuser interface device. As shown in example FIG. 3A, the media contenttransmitted by a media device (310) may be shown concurrently on userinterface device A (312) and user interface device B (314). The videocontent may be displayed by the content management device (130) at thedifferent user interface devices with different resolution and/or framerates. The content management device (130) may be a separate device orpart of media device (310) that outputs the media content.

In an embodiment, multimedia content may be displayed on user interfacedevice A (312) concurrently with related content being displayed on userinterface device B (314). Multimedia content shown on user interfacedevice A (312) may be concurrently shown on user interface device B(314) with slight modifications (e.g., related content). For example,text or other specified content may be shown at different sizes, inrelation to the underlying images or graphics, on different userinterface devices. For example, as shown in FIG. 3B, the user interfacedevice A (312) may display multimedia content overlaid with a menu (316a) for operating media device (310). The user interface device B (314)may be configured to display the same multimedia content overlaid with alarger version of the menu (316 b) in relation to the underlying mediacontent. The user interface device B (314) may be touch screen thatvisually indicates that the menu buttons are selectable by, for example,bolding the button outlines.

In an embodiment, user input may be received on user interface device B(314) and an operation related to the multimedia content displayed onthe user interface device A (312) may be performed. For example, a usermay select pause or fast forward on a menu (316 b) displayed on userinterface device B (314) while multimedia content is being concurrentlydisplayed on both of user interface device A (312) and user interfacedevice B (314). In response to receiving the user input, the multimediacontent may be paused or fast forwarded. The user input may be receivedto operate any of the devices (e.g., media devices, user interfacedevices, etc.). Input receiving on user interface device may be for thatuser interface device, for multiple user interface devices, and/or forone or more media devices.

In an embodiment, only an automatically selected spatial portion of themedia content is displayed on the secondary user interface device, whilethe media content in its entirety is concurrently displayed on theprimary user interface device. For example, as shown in example FIG. 3C,the content management device (130) may be configured to display a menuoverlaid on media content on a user interface device A (312). Thecontent management device (130) may further be configured to displayonly a spatial portion (318) of the media content that includes the menuon user interface device B (314). The menu may be generated by the mediadevice (310) that is providing the media content for display or may begenerated by and correspond to another media device. For example, themedia content may be received from a first content source (110) and themenu (e.g., with options related to brightness, color, sharpness, etc.)may be generated by a primary user interface device and overlaid on topof the media content received from the first content source (110). Thecontent management device (130) may be configured to obtain the menugenerated by the primary user interface device and display the menu on asecondary user interface that includes functionality to receive inputfrom a user.

In an embodiment, a primary user interface device is designed forstandard display at a distance from a user. For example, the primaryuser interface device may be a Liquid Crystal Display mounted on a wall.The secondary user interface device is designed as a mobile device(e.g., a cell phone, a tablet PC, a laptop, or other suitable mobiledevice). In an embodiment, the formatting of the content on thesecondary user interface (e.g., the mobile device) may vary based ondistance from the primary user interface. For example, radio frequenciesor other suitable technology may be used to determine the distancebetween the primary user interface device and the secondary user device.The text displayed on the primary user interface device may be visibleto a user, as the primary user interface may include a large displayscreen. Furthermore, assuming that the user is within an estimateddistance (e.g., 1 foot) from the secondary user interface device, anestimate of the distance between the user and primary user interfacedevice may be generated. Based on the estimated distance between theuser and primary user interface device, a size of the text as viewed bythe user on the primary user interface device may be determined. Thetext on the secondary user interface may then be enlarged or shrunk toproduce the same visual effect for the user viewing the secondary userinterface as when viewing the primary user interface device.

3.2 Content Management—Spatial and Temporal Spanning

In an embodiment, the user may use the secondary user interface deviceto zoom in on any particular spatial portion of the primary userinterface device. For example, as shown in FIG. 3D, user input may beused to select a spatial portion (320 a) of the current display on userinterface device A (312). The user input selecting the spatial portionmay be entered on the user interface device B (314) or any other device.For example, a user may first select a zoom level on user interfacedevice B (314) which results in a zoomed in version of the imagecurrently shown on user interface device A (312). Thereafter, the usermay slide a finger on user interface device B (314) signaling themovement of user selected spatial portion (320 a) in user interfacedevice A (312) to the top left corner. As a result, the zoomed in view(320 b) of the selected spatial portion (320 a) is displayed on userinterface device B (314).

In an embodiment, both user interface devices may continue toconcurrently display the media content, with the primary interfacedevice displaying the media content in entirety and the secondaryinterface device displaying a spatial portion of the media content.Furthermore, the secondary user interface device may be configured topan to another spatial portion of the media content based on user input.

In an embodiment, the secondary user interface device may be used totemporally scan the media content being displayed on the primary userinterface. For example, the content management device (130) mayinitially display the multimedia content, concurrently andsynchronously, on the primary user interface device and the secondaryuser interface at standard playback speed. The secondary user interfacedevice may then be configured based on user input to fast forward,rewind, or pause the playing of the media content while the primary userinterface device continues playing the multimedia content at standardplayback speed. The secondary user interface device may accordinglyallow a particular user to review, skip, or otherwise modify playback onthe secondary user interface device while the playing of multimediacontent on the primary user interface device is not interrupted. In anembodiment, the secondary user interface device may also include asynchronization option, which synchronizes playing of the multimediacontent on the secondary user interface device with the primary userinterface device. For example, the secondary user interface device maystart receiving the exact same video stream from a media device as thefirst user interface device and display frames synchronously with thefirst user interface device. In another example, the second userinterface device may simply resume playing of the multimedia content ata frame that is currently being displayed on the first user interfacedevice.

In an embodiment, the video streams for both the primary user interfacedevice, showing the standard video stream, and the secondary userinterface being used for temporal or spatial spanning are received froma single source of media content. For example, for spatial spanning, amedia device may output the same video stream to both the primary userinterface device and the secondary user interface device. The primaryuser interface device displays the video stream as received, while thesecondary user interface may be configured to display only a spatialportion of the video stream being received. Temporal spanning mayinvolve a media device outputting a video stream in advance to thesecondary user interface, and the secondary user interface buffering thevideo stream. The frames stored in the buffer may then be temporallypanned by a user using the secondary user interface device.

3.3 Content Management—Alternate Content

The content management device (130) may be configured to display and/orplay different version of multimedia content on different user interfacedevices (115). For example, during a showing of an R-Rated movie, thecontent management device (130) may be configured to display a censoredversion of the R-Rated movie on a primary user interface available forviewing to all audiences. The content management device (130) mayfurther be configured to concurrently display a non-censored version ofR-Rated movie on a secondary user interface device (115) (e.g., a handheld device configured for an adult viewer).

In an embodiment, the content management device (130) may receive twodifferent but related content streams for displaying on the two userinterface devices (115). In this case, the content management device(130) may simply be configured to concurrently display each contentstream on the corresponding user interface device (115). In anembodiment, the content management device (130) may receive a singlecontent stream for concurrently displaying on multiple user interfacedevices. In this case, the content management device (130) mayautomatically censor content for one user interface device while notcensoring the same content concurrently displayed on a second userinterface device.

In an embodiment, the content management device (130) may concurrentlydisplay the same video stream on multiple user interface devices whileplaying different audio streams on the multiple user interface devices.For example, the different user interface devices may play correspondingaudio in different languages. In another example, one user interfacedevice may play uncensored audio corresponding to the video stream whileanother user interface device may play censored audio corresponding tothe concurrently playing video stream.

3.4 Content Management—Informational Content/Sponsored Content

In an embodiment, the content management device (130) may be configuredto display multimedia content on a primary interface device and displayrelated content on a secondary interface device. The related content mayinclude actor information, plot synopsis, scene information, geographicinformation, etc. or any other information associated with themultimedia content. In an embodiment, the content management device(130) may receive metadata with multimedia content, where the contentmanagement device (130) is configured to display the multimedia contenton one user interface device and present the metadata on the other userinterface device. The information displayed on the secondary interfacedevice may be obtained by the content management device (130) inresponse to a request for that particular information based on a userrequest. For example, a user may request identification of a structureor a geographic scene or other information related to the media contentbeing presented. As shown in example FIG. 3E, user interface device A(312) may show the content without any additional information, whileuser interface device B (314) may be annotated with additionalinformation (322) about the structure shown in media content. A displayof other information (e.g., actor information, producer information) maybe shown on user interface device B (314) without concurrentlydisplaying the media content (e.g., the image of the house) shown inuser interface device A (312). The additional information may alsoinclude plot information (e.g., identifying characters in the plot line,good or evil characteristics, scenes, etc.) that can help viewers (e.g.,senior citizens or children) understand a complex storyline.

In an embodiment, the information presented on the secondary userinterface device may be sponsor information associated with themultimedia content displayed on the primary media interface. Theinformation concurrently presented on the secondary user interface maybe suggestions to the user for additional media content that is related(e.g., similar genre, actors, directors, producers, language, etc.) tothe multimedia content displayed on the primary user interface. In anembodiment, products and/or services featured in media content displayedon the primary user interface device may be displayed on the secondaryuser interface. For example, when a movie showing an actor using aparticular cell phone is being shown on the primary user interfacedevice, information associated with the particular cell phone may bedisplayed on the secondary user interface device.

3.5 Content Management—Alerts/Notifications

In an embodiment, the content management device (130) may be configuredto display visual alerts or play audio alerts on a first user interfacedevice but not the second user interface device. In an embodiment,multiple secondary display devices may be configured to display the samecontent as displayed on a primary display device. In addition, each ofthe multiple secondary display devices may display alerts ornotifications for a user associated with that particular secondarydisplay device. For example, a user may initially be watching content onthe primary display device. Thereafter, a user may receive apersonalized notification or alert overlaid on the same content beingdisplayed on a secondary display device (e.g., tablet PC). The user maythen switch to watching the content on the secondary display devicewhile viewing the alert or information associated with the alert. Theinformation associated with the alert may be overlaid on top of thecontent in a transparent mode, allowing the user to concurrently watchthe content. The information associated with the alert may be displayedin a second window concurrently displayed with the original content in afirst window.

In an embodiment, the content management device (130) may be configuredto display visual alerts or play audio alerts initially on the firstuser interface device and after a specified time period or othercondition, on the second user interface device. For example, if a userdoes not respond to the alert or otherwise indicate that the alert hasbeen received on the first user interface within a specified period oftime, the alert may be displayed or played on the second user interface.

In an embodiment, an alert/notification may be displayed on one userinterface device and consumed on another user interface device. Forexample, as shown in FIG. 3E, an email alert icon (324 a) indicatingthat an email has been received may be displayed on user interfacedevice A (312). The user interface device B (314) may then be used todisplay the email content (324 b), e.g., in response to user input orautomatically. The user interface device B (314) may be used toconcurrently display multimedia content while the user reads the emailmessage on user interface B (314).

3.6 Content Management—Time Delay

In an embodiment, the content management device (130) displays identicalvideo streams on multiple user interface devices with a delay in timebetween multiple displays. For example, multimedia content may bedisplayed on a secondary media device ten seconds ahead of time, incomparison to the primary media device. An adult user may then provideinput to censor or otherwise modify the content that will be displayedon the primary media device shown to all users. In an embodiment, a usermay select the time delay for displaying a video stream betweendifferent user interface devices (115). An amount of time delay may beselected automatically by the content management device (130) based onthe rating (e.g., General, Parental Guidance, Parental Guidance 13,etc.) of the media content.

3.7 Content Management—Image Layer Selection

In an embodiment, the content management device (130) may display avideo stream overlaid with a menu on a primary user interface deviceconcurrently with displaying a single frame of the video stream overlaidwith the same menu on a secondary user interface device.

For example, the content management device (130) may initially display avideo stream concurrently on both the first user interface device andthe second user interface device. In response to receiving user input atthe second user interface device (or at another device), the contentmanagement device (130) may obtain a snapshot of the frame beingdisplayed on the two user interface devices when the user input isreceived. The content management device (130) may then be configured todisplay the snapshot of the frame overlaid with a menu on the secondaryuser interface device. Furthermore, the content management device (130)may continue to display the original video stream on the primary userinterface device without any change. Alternatively, the contentmanagement device (130) may continue to display the original videostream with the menu overlaid on the original video stream, on theprimary user interface device. The secondary user interface device,displaying the single frame (e.g., snapshot) may then allow a user tosubmit input selecting an option from the menu.

In an embodiment, the content management device (130) displays multiplelayers of visual content on a user interface device. For example, asshown in FIG. 3G, the content management device (130) may display layer1 (multimedia content) (326) overlaid with layer 2 (menu) on a firstuser interface device. The menu may be overlaid on multimedia content byfirst loading a frame from the multimedia content into a frame bufferand overwriting a portion of the frame buffer with data associated withthe menu. The resulting content in the frame buffer may then bedisplayed on the user interface device A (312). The content managementdevice (130) may further be configured to display a single layer (e.g.,layer 2 (328 b)) of the available layers on user interface device B(314). Accordingly, in the above example, only the menu or only themultimedia content may be concurrently displayed on the first userinterface device and the second user interface device, while at leastone other layer is displayed on the first user interface device.

In another example, a video stream may create the illusion of a stillimage overlaid on a video stream. A spatial portion (e.g., a bottomportion) of the video stream may include a non-changing image (e.g., aDVD menu), while another spatial portion (e.g., a top portion) mayinclude a changing image (e.g., playing of a scene from a movie). Inthis illusion, although a video stream displayed on a media device maybe included in a single video content stream, the appearance of a stillimage overlaid on video content may be created on a primary userinterface device. In this example, a snapshot of a single frame from thevideo stream on the primary user interface device may be displayed on asecondary user interface device until user input is received.

3.8 Content Management—Audio

In an embodiment, the content management device (130) may manage sounddifferently for different user interface devices (115). For example, adefault setting may route all sound associated with multimedia contentbeing concurrently displayed on a primary user interface device and thesecondary interface device, to the primary user interface device. Thesecondary user interface device may be automatically activated when thesecondary user interface device is moved to a distance greater than aspecified amount from the primary user interface device. For example, auser may be watching a television program on a primary user interfacedevice (e.g., a plasma screen mounted on a wall) when a phone rings inanother room. The user may then walk away from the primary userinterface device, with the secondary user interface device (e.g., amobile device) in hand, to get the phone. When the secondary userinterface device, that is concurrently displaying the same content asthe primary user interface device, is further than a specified distancefrom the primary user interface device, the audio (and/or video) on thesecondary user interface device is automatically activated. In thisexample, the user is able to continue watching and listening to theplaying of the multimedia content in the other room. In an embodiment,the video feed may function in a similar manner. For example, thesecondary user interface device may display video stream concurrently orinstead of the primary user interface device, when the secondary userinterface device is further than a specified distance from the primaryuser interface device. In an embodiment, the video and/or audio mayswitch back to the primary user interface device when the secondary userinterface device returns within the specified distance from the primaryuser interface device.

In an embodiment, the secondary user interface device may be used bydefault for audio when the content system (100) is first turned on. Forexample, when a user turns on a content system (100) in the bedroom, theprimary user interface device may only display video with the audiobeing played on a secondary user interface device. Thereafter, the usermay provide input to the content management device (130) to turn on theaudio on the primary user interface device.

In an embodiment, a secondary user interface device may be used toprovide an additional audio stream. For example, the primary userinterface device may play a video stream with the corresponding audio.However, a particular user (e.g., a person with a hearing disability)may want a higher volume. The particular user may increase the volumefor audio played on a secondary user interface device close to theparticular user to personalize the user experience. In an embodiment,multiple secondary user interfaces may be used concurrently with theprimary user interface, where each secondary user interface isconfigured to play audio at a volume selected for that particularsecondary user interface. Using a system of multiple secondary userinterfaces with audio output, allows each user to individually select avolume level.

Although, this section presents specific examples related to audiocontent management, techniques used for visual content managementdescribed in other sections may be applicable to audio content.Furthermore, examples related to audio content management described inthe current section may applicable to visual content.

3.9 Content Management—Multiple Secondary User Interface Devices

In an embodiment, multiple secondary user interface devices may be usedconcurrently with a primary user interface device. For example, in anembodiment, media content may be displayed on the primary user interfacedevice and two or more secondary user interface devices. A user mayperform functions related to the media content on a correspondingsecondary user interface device. For example, each user may zoom in,temporally pan, spatially pan, request information, or perform any othersuitable function on a corresponding secondary user interface devicewithout affecting the primary user interface device or other userinterface devices. Each user may personalize or otherwise configure arespective secondary user interface device for alerts, notifications,messages, etc. Different secondary user interface devices may beconfigured differently and/or display different content in addition tothe common content displayed on the primary user interface device.

In an embodiment, a primary user interface device and multiple secondaryuser interface devices may be used in a gaming environment. For example,a primary user interface device may show an environment viewable by allplayers, such as a combat environment where multiple players interact.The gaming environment displayed on the primary user interface may alsoshow multiple first person views corresponding to different players. Inaddition, special weapons, abilities, functions, assets, or any othergaming attributes for a particular player may be displayed only on asecondary gaming device associated with the particular player. Inanother example, the primary user interface may display cars racingthrough a city (may include a separate view corresponding to eachplayer) and the secondary user interface for each player may includespecial speed enhancing tools, weapons, or other gaming features.

4.0 User Interface Device—Example Implementation

In an embodiment, a user interface device, described above, isimplemented as a remote control device that is communicatively coupledwith at least one of the components in the content system. In anembodiment, the remote control device may be a cellular telephone orother mobile device. In an embodiment, the remote control device may beimplemented as any mobile device or hand held device with a touch screeninterface. Although a specific structure is described herein forimplementing the user interface device (e.g., primary user interfacedevice or secondary user interface device), any other structure may beused. Any specific components described herein should not be construedas limiting the scope of the user interface device.

As shown in FIG. 2B, the remote control device (130) may becommunicatively coupled to one or more media devices through wiredand/or wireless segments, in accordance with one or more embodiments.The remote control device (130) may communicate wirelessly over one ormore of: radio waves (e.g., wi-fi signal, Bluetooth signal), infraredwaves, over any other suitable frequency in the electro-magneticspectrum, over a network connection (e.g., intranet, internet, etc.), orthrough any other suitable method.

In an embodiment, the remote control device (200) may include Read OnlyMemory (ROM) (206), a Central Processing Unit (CPU) (208), Random AccessMemory (RAM) (210), Infrared Control Unit (212), a key pad scan (214), akey pad (216), Non-Volatile Memory (NVM) (218), one or more microphones(224), gain control logic (220), analog to digital convertor (ADC)(222), a general purpose input/output (GPIO) interface (226), aspeaker/tweeter (228), a key transmitter/indicator (230), a low batteryindicator (or output signal) (232), a microphone LED (234), a radio(236), an Infrared (IR) blaster (238), a Radio Frequency (RF) Antenna(240), a QWERTY sliding keyboard (not shown), ambient noise cancellationdevice (not shown), etc. Memory on the remote control device (200)(e.g., ROM (206), RAM (210), or NVM (218)) may include control codesand/or key codes for one or more media devices (e.g., media device A(100) or media device B (120)). The memory may include aRun-Length-Limited (RLL) waveform table.

In an embodiment, the low battery indicator (232) may correspond to avisual indication (e.g., LED light) on the remote control device (200)for a low battery level. In an embodiment, the low battery indicator(232) may represent a signal output for display on a screen other thanon the remote control device (200). In an embodiment, the low batterycode is sent with a standard command. For example, when a channelselection is made on the remote control device (200), the command forthe channel selection device is piggy backed with a low battery signalfor display on a display screen.

In an embodiment, a microphone (224) may be located anywhere on theremote control device (200) (e.g., one or more microphones (224) may belocated at the ends of a remote control device (200)). If multiplemicrophones are available and turned on, the multiple microphones may beused to obtain user input. In an embodiment, one of the multiplemicrophones may be used for noise cancellation/optimizationmanipulations. A single audio stream may be determined from multipleinput audio streams by the remote control device (200) or by a mediadevice which receives the multiple audio streams from the remote controldevice (200).

In an embodiment, the remote control device (200) may include aproximity sensor (not shown) to detect presence of a user within aspecified distance of the remote control device (200) even before theuser presses a button on the remote control device (200). For example,the remote control device (200) may operate in a low power state until auser is detected. Once a user is detected, the remote control device(200) may operate in a normal power state or a high power state. Theremote control device (200) may be configured to turn on keypad lightsas soon as a user is detected. In an embodiment, the proximity sensormay be based on capacitive coupling to detect a user near the remotecontrol device (200).

In an embodiment, the remote control device (200) includes one or moredisplays (242). The displays may be touch screen displays that includefunctionality to receive user input by a user touching the displayscreen. The display (242) may be used as a secondary display of asecondary interface device (e.g., the remote control device (200)).Content on display (242) may be related to content displayed anotherdisplay device (e.g., on a primary user interface). Content on thedisplay (242) and content on the primary user interface may both betransmitted from a single media device or media management device.

5.0 Command Execution—Example Implementation

The following example, describes one possible method of communicationbetween a user interface device and other devices within the system. Theset of devices and the steps performed by the set of devices should notbe construed as limiting in scope as other variations of the set ofdevices and the steps performed may be implemented with otherembodiments.

A command is received from a user interface device for operating atarget media device. The user interface device requests informationassociated with the command from a second media device. The userinterface device may request information about the command itself. Forexample, the user interface device may request the actual signal, fromthe second media device, corresponding to the command for transmissionto the target media device. The user interface device may request aportion of the actual signal from the second media device. For example,the user interface device may request only a device code for the targetmedia device or other identification of the target media device to usein a signal (e.g., an infrared signal) to send to the target mediadevice.

The user interface device may request state information about the systemmaintained by the second media device. For example, the second mediadevice may function as a management device and maintain current stateinformation about the system. Examples of state information include thecurrent display information such as interfaces displayed, selectionsdisplayed to a user, media content being played, media device providinginput to the current display, channel selected, etc. State informationmay include current configuration settings such as volume, brightness,tint, color, user preferences, etc. State information may include mediadevice information such recordings stored on media devices, recordingschedules, viewing/recording history, etc.

The second media device may transmit information associated with thecommand to the user interface device. The second media device maytransmit any of the information requested by the user interface device,as described above. For example, the second media device may transmit acode of the target media device or the actual signal for the userinterface device to transmit to the target media device. The secondmedia device may transmit any of the information, described above,without receiving a specific request for the information. For example,the second media device may periodically update the user interfacedevice with state information. The second media device may provide theinformation to the user interface device in response to detecting a lowusage level of the user interface device or the second media device. Forexample, a number of processor operations over a period of time may bemonitored to determine a usage level and thereafter once a thresholdlevel indicative of low usage is detected, the second media device maytransmit state information updates to the user interface device.

The user interface device may determine the signal to be sent to thetarget media device based on the command received from the user and/orthe information received from the second media device. Determining thesignal may be as simple as receiving all the information including thesignal from the second media device. Determining the signal may includedetermining an operation code based on the command received by accessinga locally stored table which maps commands received to operation codes.Determining the signal may include combining the operation code withinformation identifying the target media device received from the secondmedia device. Based on the current display and the command received(e.g., an up button or a select button), the user interface device maydetermine the signal to be transmitted to the target media device.

Another example may involve a user interface device communicatingdirectly with a target media device without interaction with otherdevices. Yet another example may involve a user interface devicecommunication information associated with a command to a second mediadevice which then communicates with the target media device to perform afunction based on the command.

6.0 Hardware Overview

According to one embodiment, the techniques described herein areimplemented by one or more special-purpose computing devices. Thespecial-purpose computing devices may be hard-wired to perform thetechniques, or may include digital electronic devices such as one ormore application-specific integrated circuits (ASICs) or fieldprogrammable gate arrays (FPGAs) that are persistently programmed toperform the techniques, or may include one or more general purposehardware processors programmed to perform the techniques pursuant toprogram instructions in firmware, memory, other storage, or acombination. Such special-purpose computing devices may also combinecustom hard-wired logic, ASICs, or FPGAs with custom programming toaccomplish the techniques. The special-purpose computing devices may bedesktop computer systems, portable computer systems, handheld devices,networking devices or any other device that incorporates hard-wiredand/or program logic to implement the techniques.

For example, FIG. 4 is a block diagram that illustrates a System 400upon which an embodiment of the invention may be implemented. System 400includes a bus 402 or other communication mechanism for communicatinginformation, and a hardware processor 404 coupled with bus 402 forprocessing information. Hardware processor 404 may be, for example, ageneral purpose microprocessor.

System 400 also includes a main memory 406, such as a random accessmemory (RAM) or other dynamic storage device, coupled to bus 402 forstoring information and instructions to be executed by processor 404.Main memory 406 also may be used for storing temporary variables orother intermediate information during execution of instructions to beexecuted by processor 404. Such instructions, when stored in storagemedia accessible to processor 404, render System 400 into aspecial-purpose machine that is customized to perform the operationsspecified in the instructions.

System 400 further includes a read only memory (ROM) 408 or other staticstorage device coupled to bus 402 for storing static information andinstructions for processor 404. A storage device 410, such as a magneticdisk or optical disk, is provided and coupled to bus 402 for storinginformation and instructions.

System 400 may be coupled via bus 402 to a display 412, such as acathode ray tube (CRT), for displaying information to a computer user.An input device 414, including alphanumeric and other keys, is coupledto bus 402 for communicating information and command selections toprocessor 404. Another type of user input device is cursor control 441,such as a mouse, a trackball, or cursor direction keys for communicatingdirection information and command selections to processor 404 and forcontrolling cursor movement on display 412. This input device typicallyhas two degrees of freedom in two axes, a first axis (e.g., x) and asecond axis (e.g., y), that allows the device to specify positions in aplane.

System 400 may implement the techniques described herein usingcustomized hard-wired logic, one or more ASICs or FPGAs, firmware and/orprogram logic which in combination with the System causes or programsSystem 400 to be a special-purpose machine. According to one embodiment,the techniques herein are performed by System 400 in response toprocessor 404 executing one or more sequences of one or moreinstructions contained in main memory 406. Such instructions may be readinto main memory 406 from another storage medium, such as storage device410. Execution of the sequences of instructions contained in main memory406 causes processor 404 to perform the process steps described herein.In alternative embodiments, hard-wired circuitry may be used in place ofor in combination with software instructions.

The term “storage media” as used herein refers to any media that storedata and/or instructions that cause a machine to operation in a specificfashion. Such storage media may comprise non-volatile media and/orvolatile media. Non-volatile media includes, for example, optical ormagnetic disks, such as storage device 410. Volatile media includesdynamic memory, such as main memory 406. Common forms of storage mediainclude, for example, a floppy disk, a flexible disk, hard disk, solidstate drive, magnetic tape, or any other magnetic data storage medium, aCD-ROM, any other optical data storage medium, any physical medium withpatterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, NVRAM, anyother memory chip or cartridge.

Storage media is distinct from but may be used in conjunction withtransmission media. Transmission media participates in transferringinformation between storage media. For example, transmission mediaincludes coaxial cables, copper wire and fiber optics, including thewires that comprise bus 402. Transmission media can also take the formof acoustic or light waves, such as those generated during radio-waveand infra-red data communications.

Various forms of media may be involved in carrying one or more sequencesof one or more instructions to processor 404 for execution. For example,the instructions may initially be carried on a magnetic disk or solidstate drive of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over atelephone line using a modem. A modem local to System 400 can receivethe data on the telephone line and use an infra-red transmitter toconvert the data to an infra-red signal. An infra-red detector canreceive the data carried in the infra-red signal and appropriatecircuitry can place the data on bus 402. Bus 402 carries the data tomain memory 406, from which processor 404 retrieves and executes theinstructions. The instructions received by main memory 406 mayoptionally be stored on storage device 410 either before or afterexecution by processor 404.

System 400 also includes a communication interface 418 coupled to bus402. Communication interface 418 provides a two-way data communicationcoupling to a network link 420 that is connected to a local network 422.For example, communication interface 418 may be an integrated servicesdigital network (ISDN) card, cable modem, satellite modem, or a modem toprovide a data communication connection to a corresponding type oftelephone line. As another example, communication interface 418 may be alocal area network (LAN) card to provide a data communication connectionto a compatible LAN. Wireless links may also be implemented. In any suchimplementation, communication interface 418 sends and receiveselectrical, electromagnetic or optical signals that carry digital datastreams representing various types of information.

Network link 420 typically provides data communication through one ormore networks to other data devices. For example, network link 420 mayprovide a connection through local network 422 to a host computer 424 orto data equipment operated by an Internet Service Provider (ISP) 424.ISP 424 in turn provides data communication services through the worldwide packet data communication network now commonly referred to as the“Internet” 428. Local network 422 and Internet 428 both use electrical,electromagnetic or optical signals that carry digital data streams. Thesignals through the various networks and the signals on network link 420and through communication interface 418, which carry the digital data toand from System 400, are example forms of transmission media.

System 400 can send messages and receive data, including program code,through the network(s), network link 420 and communication interface418. In the Internet example, a server 430 might transmit a requestedcode for an application program through Internet 428, ISP 424, localnetwork 422 and communication interface 418.

The received code may be executed by processor 404 as it is received,and/or stored in storage device 410, or other non-volatile storage forlater execution.

7.0 Extensions and Alternatives

In the foregoing specification, embodiments of the invention have beendescribed with reference to numerous specific details that may vary fromimplementation to implementation. Thus, the sole and exclusive indicatorof what is the invention, and is intended by the applicants to be theinvention, is the set of claims that issue from this application, in thespecific form in which such claims issue, including any subsequentcorrection. Any definitions expressly set forth herein for termscontained in such claims shall govern the meaning of such terms as usedin the claims. Hence, no limitation, element, property, feature,advantage or attribute that is not expressly recited in a claim shouldlimit the scope of such claim in any way. The specification and drawingsare, accordingly, to be regarded in an illustrative rather than arestrictive sense.

1. A method, comprising: sending multimedia content to a first deviceand a second device for display on the first device and the seconddevice; selecting one of the first device and the second device todisplay additional content based on a type of the additional content ora source of the additional content; sending the additional content tothe selected device for display on the selected device; wherein themethod is performed by one or more devices.
 2. The method as recited inclaim 1, wherein the additional content is overlaid on the multimediacontent for display on the selected device.
 3. The method as recited inclaim 1, wherein the additional content is displayed in response toreceiving a request for the additional content.
 4. The method as recitedin claim 1, wherein the additional content comprises an alertnotification.
 5. The method as recited in claim 4, wherein the alertnotification is displayed on the first device and detailed informationassociated with the alert notification is concurrently displayed on thesecond device.
 6. The method as recited in claim 4, wherein theselecting step comprises selecting the second device, and furthercomprising: subsequent to a particular time interval of not receivinguser input at the second device in response to displaying the alertnotification on the second device: displaying the alert notification onthe first device.
 7. The method as recited in claim 1, wherein theadditional content comprises a first message associated with a webservice.
 8. The method as recited in claim 7, further comprisingdisplaying a text message on the second device based on the firstmessage associated with the web service, wherein the second devicecomprises a cellular telephone.
 9. The method as recited in claim 7,further comprising receiving a second message from the second devicebased on user input and sending information associated with the secondmessage to the web service.
 10. The method as recited in claim 9,wherein the first message is associated with a bill and the secondmessage is associated with payment authorization for the bill.
 11. Anon-transitory computer readable storage medium storing instructions,which when executed by one or more processors performs steps comprising:sending multimedia content to a first device and a second device fordisplay on the first device and the second device; selecting one of thefirst device and the second device to display additional content basedon a type of the additional content or a source of the additionalcontent; sending the additional content to the selected device fordisplay on the selected device.
 12. The computer readable storage mediumas recited in claim 11, wherein the additional content is overlaid onthe multimedia content for display on the selected device.
 13. Thecomputer readable storage medium as recited in claim 11, wherein theadditional content is displayed in response to receiving a request forthe additional content.
 14. The computer readable storage medium asrecited in claim 11, wherein the additional content comprises an alertnotification.
 15. The computer readable storage medium as recited inclaim 14, wherein the alert notification is displayed on the firstdevice and detailed information associated with the alert notificationis concurrently displayed on the second device.
 16. The computerreadable storage medium as recited in claim 14, wherein the selectingstep comprises selecting the second device, and the steps furthercomprising: subsequent to a particular time interval of not receivinguser input at the second device in response to displaying the alertnotification on the second device: displaying the alert notification onthe first device.
 17. The computer readable storage medium as recited inclaim 11, wherein the additional content comprises a first messageassociated with a web service.
 18. The computer readable storage mediumas recited in claim 17, the steps further comprising displaying a textmessage on the second device based on the first message associated withthe web service, wherein the second device comprises a cellulartelephone.
 19. The computer readable storage medium as recited in claim17, the steps further comprising receiving a second message from thesecond device based on user input and sending information associatedwith the second message to the web service.
 20. The computer readablestorage medium as recited in claim 19, wherein the first message isassociated with a bill and the second message is associated with paymentauthorization for the bill.
 21. An apparatus comprising: one or moreprocessors; logic configured to perform steps comprising: sendingmultimedia content to a first device and a second device for display onthe first device and the second device; selecting one of the firstdevice and the second device to display additional content based on atype of the additional content or a source of the additional content;sending the additional content to the selected device for display on theselected device.
 22. The apparatus as recited in claim 11, wherein theadditional content is overlaid on the multimedia content for display onthe selected device.
 23. The apparatus as recited in claim 11, whereinthe additional content is displayed in response to receiving a requestfor the additional content.
 24. The apparatus as recited in claim 11,wherein the additional content comprises an alert notification.
 25. Theapparatus as recited in claim 14, wherein the alert notification isdisplayed on the first device and detailed information associated withthe alert notification is concurrently displayed on the second device.26. The apparatus as recited in claim 14, wherein the selecting stepcomprises selecting the second device, and the steps further comprising:subsequent to a particular time interval of not receiving user input atthe second device in response to displaying the alert notification onthe second device: displaying the alert notification on the firstdevice.
 27. The apparatus as recited in claim 11, wherein the additionalcontent comprises a first message associated with a web service.
 28. Theapparatus as recited in claim 17, the steps further comprisingdisplaying a text message on the second device based on the firstmessage associated with the web service, wherein the second devicecomprises a cellular telephone.
 29. The apparatus as recited in claim17, the steps further comprising receiving a second message from thesecond device based on user input and sending information associatedwith the second message to the web service.
 30. The apparatus as recitedin claim 19, wherein the first message is associated with a bill and thesecond message is associated with payment authorization for the bill.